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(54) Method and system for conducting a multimedia phone call 



(57) A method and apparatus tor conducting a mul- 
timedia telephone call. The apparatus carries out the 
method by establishing a telephone call between first 
and second customer premises equipment and gener- 
ating, at the first customer premises equipment, a fre- 
quency shift keyed message containing a network re- 
source locator. The FSK message is then transmitted 
from the first customer premises equipment to the sec- 
ond customer premises equipment during the telephone 
call and is received at the second customer premises 
equipment. 
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Description 

FIELD OF THE INVENTION 

[0001] This invention relates to a method and system 
for conducting a multimedia telephone call, and more 
particularly to a system which facilitates a voice tele- 
phone call while navigating to a network resource spec- 
ified by a far end telephone. 

BACKGROUND OF THE INVENTION 

[0002] With the advent of the world-wide web, various 
websites are available as network resources from which 
a user may download and view advertising and informa- 
tion relating to goods and services offered by a compa- 
ny. In order to do this, the user must first be provided 
with the identity of the website to which the user should 
navigate. 

[0003] Once the user has navigated to a particular 
website, the user's interaction with the website is, es- 
sentially, one of machine interaction in the sense that 
the user makes choices as to which displays or portions 
of the display he/she wishes to navigate to. Often, web- 
sites will provide instructions as to how to navigate about 
the website. Such navigation may include indexes to 
other pages available at the website. 
[0004] Unfortunately, however, users are not always 
aware of available websites providing information of in- 
terest, nor are they aware of all of the various content 
that may be provided in some of the more complex web- 
sites. Often to navigate about such content, a user must 
investigate all links which appear to be of interest. Often, 
however, identifications of links do not always convey 
the content of the resource associated with the link and, 
therefore, the user may be directed to links which are 
really not of interest. In any event, the choice of links to 
navigate to, depend upon the user's interpretation of the 
names of the links. 

[0005] .What would be desirable, therefore, is a way 
of identifying to a user links which may be important to 
the user's particular needs, based on an interpretation 
of the user's needs provided by the provider. In addition, 
the ability to converse with a remote party observing the 
same image of goods or services or content, at the same 
time, would be desirable. The present invention ad- 
dresses this need. 

SUMMARY OF THE INVENTION 

[0006] In accordance with one aspect of the invention, 
there is provided a method of conducting a multimedia 
telephone call, the method including the steps of: 

a) establishing a telephone call between first and 
second customer premises equipment; 

b) generating at the first customer premises equip- 



ment a frequency shift keyed message containing 
a network resource locator; 

c) transmitting the FSK message from the first cus- 
s tomer premises equipment to the second customer 

premises equipment during the telephone call; and 

d) receiving the FSK message at the second cus- 
tomer premises equipment. 

10 

[0007] Preferably, the method involves the steps of 
extracting the network resource locator from the FSK 
message, establishing a connection to a network re- 
source indicated by the network resource locator. De- 
is sirably, the connection is established using a second tel- 
ephone line. 

[0008] Preferably, the connection to the network re- 
source is established while maintaining the telephone 
call active. 

20 [0009] The method further includes the step of down- 
loading a file from the network resource identified by the 
network resource locator and producing a display image 
at the second customer premises equipment, from the 
file, while displaying an associated image at the first cus- 

25 tomer premises equipment. 

[0010] More particularly, the method preferably in- 
cludes the step of producing a display image at the sec- 
ond customer premises equipment while displaying an 
associated image at the first customer premises equip- 

30 ment, while maintaining the telephone call to enable a 
user of the second customer premises equipment to 
view images designated by a user of the first customer 
premises equipment, while conducting a telephone con- 
versation with the user of the first customer premises 

35 equipment. 

[0011] It is desirable that there is maintained a list of 
network resource locators at the first customer premises 
equipment and that the network resource locators are 
used in respective FSK messages. 

40 [0012] Desirably, the-method includes ihe step of as- 
sociating image files with respective network resource 
locators and producing display images corresponding 
to respective network resource locators when an FSK 
message is generated. 

45 [0013] In accordance with another aspect of the in- 
vention, there is provided a system for conducting a mul- 
timedia telephone call. The system includes first and 
second customer premises equipment and provisions 
which are disposed at at least one of the first and second 

50 customer premises equipment for establishing a tele- 
phone call on a first telephone line, between the first and 
second customer premises equipment. The system fur- 
ther includes provisions for generating at the first cus- 
tomer premises equipment a frequency shift keyed mes- 

55 sage containing a network resource locator and provi- 
sions for transmitting the FSK message from the first 
customer premises equipment to the second customer 
premises equipment during the telephone call. Provi- 
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sions are provided at the second customer premises 

equipment, for receiving the FSK message. 

[0014] Preferably, the apparatus includes provisions 

for extracting the network resource locator from the FSK 

message. 

[0015] Desirably, the apparatus includes provisions 
for establishing a connection to a network resource in- 
dicated by the network resource locator. 
[0016] Preferably, the apparatus includes provisions 
for establishing the connection to the network resource 
on a second telephone line. 

[0017] Desirably, the apparatus includes provisions 
for establishing the connection to the network resource 
while maintaining the telephone call active on the first 
telephone line. 

[0018] Preferably, the apparatus includes provisions 
for downloading a file from the network resource identi- 
fied by the network resource locator. 
[0019] Desirably, the apparatus includes provisions 
for producing a display image at the second customer 
premises equipment, from the file, while displaying an 
associated image at the first customer premises equip- 
ment. 

[0020] Preferably, the apparatus includes provisions 
for producing a display image at the second customer 
premises equipment while displaying an associated im- 
age at the first customer premises equipment, while 
maintaining the telephone call on the first telephone line 
to enable a user of the second customer premises 
equipment to view images designated by a user of the 
first customer premises equipment, while conducting a 
telephone conversation with the user of the first custom- 
er premises equipment. 

[0021] Desirably, the apparatus includes provisions 
for maintaining a list of network resource locators at the 
first customer premises equipment and selectively using 
the network resource locators in respective FSK mes- 
sages. 

[0022] Preferably, the apparatus includes provisions 
for associating image files with respective network re- 
source locators and producing display images at the first 
customer premises equipment, the display images cor- 
responding to respective network resource locators 
when an FSK message is generated. 
[0023] In accordance with another aspect of the in- 
vention, there is provided a system for conducting a mul- 
timedia telephone call, the system including first and 
second customer premises equipment operable to es- 
tablish a telephone call between each other on a first 
telephone line, a processor at the first customer premis- 
es equipment, for generating at the first customer 
premises equipment a frequency shift keyed message 
containing a network resource locator, a transmitter for 
transmitting the FSK message from the first customer 
premises equipment to the second customer premises 
equipment during the telephone call and a receiver at 
the second customer premises equipment for receiving 
the FSK message. 



[0024] Desirably, the apparatus includes a processor 
at the second customer premises equipment for extract- 
ing the network resource locator from the FSK message. 

5 [0025] Preferably, the apparatus includes a processor 
and a browser at the second customer premises equip- 
ment for establishing a connection to a network re- 
source indicated by the network resource locator. 
[0026] Desirably, the apparatus includes a modem for 

io establishing the connection to the network resource on 
a second telephone line. 

[0027] Preferably, the processor at the second cus- 
tomer premises equipment is operable to establish the 
connection to the network resource while maintaining 

15 the telephone call active on the first telephone line. 
[0028] Desirably, the processor and the browser at 
the second customer premises equipment is operable 
to download a file from the network resource identified 
by the network resource locator 

20 [0029] Preferably, the apparatus includes first and 
second displays respective at the first and second cus- 
tomer premises equipment and desirably, the processor 
at the second customer premises equipment is operable 
to cooperate with the second display to produce a dis- 

25 play image at the second customer premises equip- 
ment, from the file, while the processor at the first cus- 
tomer premises equipment cooperates with the first dis- 
play to display an associated image at the first customer 
premises equipment. 

30 [0030] Preferably, the apparatus includes a first proc- 
essor and memory at the first customer premises equip- 
ment for maintaining a list of network resource locators 
at the first customer premises equipment, the processor 
being programmed to selectively include the network re- 

35 source locators in respective FSK messages. 

[0031] Desirably, the first processor is programmed to 
associate image files with respective network resource 
locators and produce display images at the first custom- 
er premises equipment, the display images correspond- 

40 . ing to respective network resource locators included in 
the FSK messages, as the FSK messages are generat- 
ed. 

[0032] In accordance with another aspect of the in- 
vention, there is provided a method for dynamically dis- 
<5 patching network resource locators. The method in- 
cludes the steps of: 

a) establishing a telephone call between first and 
second customer premises equipment; 

50 

b) generating at the first customer premises equip- 
ment a frequency shift keyed message containing 
the network resource locator; and 

55 c) transmitting the FSK message from the first cus- 
tomer premises equipment to the second customer 
premises equipment during the telephone call. 
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[0033] Preferably, the method includes the steps of 
maintaining a list of network resource locators at the first 
customer premises equipment and selectively using the 
network resource locators in respective FSK messages. 
[0034] Desirably, the method includes the steps of as- 
sociating image files with respective network resource 
locators and producing display Images corresponding 
to respective network resource locators when an FSK 
message is generated. 

[0035] In accordance with another aspect of the in- 
vention, there is provided a system for dynamically dis- 
patching network resource locators. The system in- 
cludes first and second customer premises equipment 
and provisions which are disposed at at least one of the 
first and second customer premises equipment for es- 
tablishing a telephone call on a first telephone line, be- 
tween the first and second customer premises equip- 
ment, provisions for generating at the first customer 
premises equipment a frequency shift keyed message 
containing the network resource locator, and provisions 
for transmitting the FSK message from the first custom- 
er premises equipment to the second customer premis- 
es equipment during the telephone call. 
[0036] Preferably, the apparatus includes provisions 
for maintaining a list of network resource locators at the 
first customer premises equipment and selectively using 
the network resource locators in respective FSK mes- 
sages. 

[0037] Desirably, the apparatus includes provisions 
for associating image files with respective network re- 
source locators and for producing display images at the 
first customer premises equipment, the display images 
corresponding to respective network resource locators 
when an FSK message is generated. 
[0038] In accordance with another aspect of the in- 
vention, there is provided a system for dynamically dis- 
patching network resource locators. The system in- 
cludes first and second customer premises equipment 
operable to establish a telephone call between each oth- 
er on a first telephone line, a processor at the first, cus- 
tomer premises equipment, for generating at the first 
customer premises equipment a frequency shift keyed 
message containing the network resource locator and 
a transmitter for transmitting the FSK message from the 
first customer premises equipment to the second cus- 
tomer premises equipment during the telephone call. 
[0039] Preferably, the apparatus includes a first proc- 
essor and memory at the first customer premises equip- 
ment for maintaining a list of network resource locators 
at the first customer premises equipment, the processor 
being programmed to selectively include the network re- 
source locators in respective FSK messages. 
[0040] Desirably, the first processor is programmed to 
associate image files with respective network resource 
locators and produce display images at the first custom- 
er premises equipment, the display images correspond- 
ing to respective network resource locators included in 
the FSK messages, as the FSK messages are generat- 



BRIEF DESCRIPTION OF THE DRAWINGS 

5 [0041] In drawings which illustrate embodiments of 
the invention, 

Figure 1 is a schematic diagram of a system, ac- 
cording to a first embodiment of the inven- 
tion; 

is a schematic representation of an FSK 
message transmitted from a first custom- 
er premises equipment to a second cus- 
tomer premises equipment shown in Fig- 
ure 1; 

is a schematic representation of a tele- 
phone according to the first embodiment 
of the invention; - - - 

is a block diagram of a central office line 
interface in the second telephone shown 
in Figure 3; 

is a tree diagram of a directory structure 
of file directories within the second tele- 
phone; 

is a schematic representation of storage 
areas in the RAM shown in Figure 3; 

is a schematic representation of a pro- 
gram architecture in a processor in the 
first telephone shown in Figure 3; 

is a flowchart of an event parser thread 
run by a system processor of the first tel- 
ephone shown in Figure 3; 

is a flowchart of a main event handler 
thread run by the system processor; 

Figure 10 is a flowchart of an event dispatcher run 
45 by the system processor; 

Figure 1 1 is a flowchart of the browser thread run by 
the system processor; and 

so Figure 12 is a flowchart of a network driver run by 
the system processor. 

DETAILED DESCRIPTION 



[0042] Referring to Figure 1 , a system for conducting 
a multimedia phone call is shown generally at 10. The 
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apparatus includes a first customer premises equipment 
shown generally at 12 and second customer premises 
equipment shown generally at 20. In this embodiment, 
the first customer premises equipment 12 is connected 
to a central office switch 22 by a first telephone line 16. 
Similarly, the second customer premises equipment 20 
is connected to the central office switch 22 by second 
and third telephone lines 18 and 28. Also connected to 
the central office switch 22 is an internet services pro- 
vider 24 for providing communication facilities with the 
World Wide Web shown generally at 23. The world wide 
web 23 includes a first content provider 26 having a web 
server 350 and a database of hypertext markup lan- 
guage (HTML) content files 352 which may be served 
to the World Wide Web through the web server 350. 
[0043] Still referring to Figure 1, in this embodiment 
the customer premises equipment 12 includes a person- 
al computer 30 and a conventional telephone 32. The 
personal computer 30 includes a microprocessor 34, 
memory 36, a frequency shift keyed data transmitter 38, 
a mute circuit 40, a keyboard 42 and a display 44, all in 
communication with the microprocessor 34. The con- 
ventional telephone 32 acts as means at the first cus- 
tomer premises equipment for establishing a telephone 
call on a first telephone line, between the first and sec- 
ond customer premises equipment. 
[0044] The frequency shift keyed data transmitter 38 
and mute circuit 40 are connected to the telephone line 
16 from the central office switch 22 and the mute circuit 
40 is connected in series with the telephone 32 and the 
telephone line 16 to selectively mute signals appearing 
on the telephone line, from the telephone 32. The mi- 
croprocessor 34 controls the mute circuit 40 and the 
FSK transmitter 38 such that when FSK data is being 
transmitted by the FSK transmitter 38, the mute circuit 
40 is rendered operational to mute FSK signals from the 
telephone 32. 

[0045] In this embodiment, the memory 36 is loaded 
with a list of uniform resource locators identifying re- 
spective pages at the first content provider 56, and im-. 
age files associated with respective uniform resource lo- 
cators. The microprocessor is programmed to receive 
input from the keyboard 42 identifying a desired URL 
and to generate an FSK message having a format as 
shown in Figure 2 and to display an image file associ- 
ated with a URL in a respectively transmitted FSK mes- 
sage. Thus the microprocessor acts as means for asso- 
ciating image files with respective network resource lo- 
cators and producing display images at the first custom- 
er premises equipment, the display images correspond- 
ing to respective network resource locators when an 
FSK message is generated. 

[0046] In general, the personal computer at the first 
customer premises equipment acts as a system for dy- 
namically dispatching network resource locators. 



Figure 2 

[0047] Referring to Figure 2 the FSK message in- 
cludes a message type field 72, a total length field 74, 

5 a cpe id field 76, a parameter type field 78, a parameter 
length field 80 and a parameter data field 82. The mes- 
sage type field identifies the message as an FSK mes- 
sage containing a URL identifying a network resource 
from which a page can be downloaded. Predefined 

10 codes are used to identify predefined message types. 
The total length field 74 is used to hold a value repre- 
senting the total length of the FSK message. The CPE 
ID field 76 is loaded with a value representing a CPE ID, 
which in this embodiment is zero. The parameter type 

is field 78 is used to identify the type of data stored in the 
parameter data field 82 and, in this embodiment, the pa- 
rameter type field stores a predefined code representing 
that the contents of the parameter data field 82 hold a 
URL. The parameter length field 80 is used to identify 

20 the length of the parameter data field 82. The parameter 
data field 82 is, in this embodiment, used to hold a uni- 
form resource locator identified by input received at the 
keyboard 42 and copied from the memory 36. 

25 Figure 3 

[0048] Referring to Figure 3, the telephone is shown 
generally at 20 and includes a system microprocessor 
90; a telephony microcontroller 92, and a central office 

30 line interface 94 in communication with both the system 
microprocessor 90 and the telephony microcontroller 
92. The system microprocessor is connected to a key- 
board interface 96 and a keyboard 98, for receiving user 
input for commanding the system microprocessor 90 to 

35 effect certain functionality. The system microprocessor 
is further connected to a display interface 100 which is 
further connected to a display 1 02, for displaying output 
to a user. 

[0049] The telephony microcontroller 92 is in commu- 
w nicatlon with the system microprocessor 90 by a plurality 
of signal lines 104 and is further in communication with 
a handset interface 105 for controlling signals to and 
from a conventional handset 106 and a hands-free in- 
terface 107 for providing conventional speakerphone 
45 functionality to the telephone. 

[0050] The telephony microcontroller 92 is further in 
communication with a dial pad 108 of the conventional 
type, for enabling a user to dial telephone numbers at 
the telephone. 

50 

Figure 4 

[0051] Referring to Figure 4, the central office line in- 
terface is shown in greater detail at 94. The central office 
55 |jne interface has first and second telephone line inputs 
110 and 112 which are connected to the second and 
third telephone lines 18 and 28 respectively, from the 
central office switch 22 shown in Figure 1. 



5 



9 



EP0 971 513 A2 



10 



[0052] The first line input 110 is connected to a line 1 
modem terminal 114, a two to four wire interface termi- 
nal 116 and an on-hook extension in use detector 118. 
The modem terminal 114 is connected to a line selector 
switch shown generally at 120 having a selector terminal s 
122 and a line 2 modem terminal 124. The line selector 
switch 120 is controlled by a line selector control circuit 
126 to connect the selector terminal 122 to either the 
line 2 modem terminal 124 or the line 1 modem terminal 
114. The selector terminal 122 is further connected to a 10 
modem hook switch 1 28 which is controlled by a modem 
130 in communication with the system microprocessor 
90, shown in Figure 3. 

[0053] Referring back to Figure 4, the modem hook 
switch 128 is connected to a data access arrangement is 
132 which isolates the modem from the telephone line 
and provides a DC path for seizing the line. The data 
access arrangement 132 is in communication with the 
modem 1 30, to provide analog signals to and from the 
modem 130, for communication to the . system micro- 20 
processor. 

[0054] The central office line interface further includes 
a line hook switch 134 and a two wire to four wire hybrid 
circuit 136. The hook switch 134 is controlled by a line 
hook switch control circuit 1 38 which opens and closes 25 
the hook switch 1 34 thereby connecting and disconnect- 
ing the second telephone line 18 to and from the two to 
four wire hybrid circuit 136. 

[0055] The two to four wire hybrid circuit 136 has a 
transmit input 1 40 and a receive output 1 42 and an off- 30 
hook detect signal output 1 44. The transmit input 1 40 is 
connected to a DTMF generator 146 and a transmit out- 
put of an audio circuit 148. The DTMF generator is op- 
erable to provide tones to the transmit input 140 to pro* 
vide DTMF tones on the third telephone line 18. 35 
[0056] The receive output 142 of the two to four wire 
hybrid circuit 136 is connected to a receive input of the 
audio circuit 148, and is further connected to an FSK 
receiver 150 and a caller alerting signal receiver 152. 
_ [0057] The off-hook PU.detect output 144 of the two 40 
to four wire hybrid circuit 1 36 is connected to an off -hook 
EIU detect circuit 154 which provides a signal indicating 
whether or not an extension is in use on the second tel- 
ephone line 18, at the same time that the hook switch 
134 is closed. 45 
[0058] The on-hook extension in use detector 118 
monitors the second telephone line 18 to provide a sig- 
nal indicating whether or not an extension is in use on 
the first telephone line, when the hook switch 134 is in 
the open position. so 
[0059] The line selector control circuit 126, the line 
hook switch circuit 1 38 and the DTMF generator 1 46 are 
operable to receive signals from the telephony micro- 
controller 92 shown in Figure 3, to control the state of 
the line selector switch 120, to control the state of the 55 
line hook switch 1 34 and to provide DTMF signals to the 
two to four wire interface for transmission on the second 
telephone line 18, respectively. The audio circuit 148 is 



in bi-directional communication with the telephony mi- 
crocontroller which effectively provides an audio path, 
with audio signal processing to the handset interface 
105 or hands-free interface 107 shown in Figure 3. 
[0060] The FSK receiver 1 50, caller alerting signal re- 
ceiver 152, off-hook EIU detect circuit 154 and on-hook 
extension in use detector 118 provide FSK, CAS, off- 
hook EIU and on-hook EIU signals respectively to the 
telephony microcontroller. 

[0061] Referring back to Figure 3, the system micro- 
processor 90 is further in communication with FLASH 
memory 156 and random access memory (RAM) 158. 

Figure 5 

[0062] Referring to Figure 5, the FLASH memory 
holds a directory structure as shown generally at 161. 
The directory structure includes a base directory having 
a hardware abstraction layer file 163, an operating sys- 
tem file 165, a system subdirectory 167, a data directory 
169, a Java subdirectory 171 and a temporary subdirec- 
tory 173. 

[0063] The hardware abstraction layer file 163 in- 
cludes basic boot commands for booting up the system 
processor and for directing it to load the operating sys- 
tem file 165 into RAM, where it is run by the processor. 
[0064] The operating system file 165 contains oper- 
ating system commands for establishing basic opera- 
tion of the system microprocessor 90. The operating 
system commands include a Board Support Package 
(BSP) and drivers which direct the processor to interact 
with the central office line interface 94, the telephony 
microcontroller 92, the keyboard interface 96 and the 
display interface 100 shown in Figure 3.. In this embod- 
iment, the operating system is known as VXWorks (tm) 
provided by Windriver of Alameda, California. The op- 
erating system maintains a routing table 155 in the RAM 
1 58 shown in Figure 3 for maintaining details of connec- 
tions established through the central office line interface 
94 on the third subscriber-line 28 shown in Figure 4. 
[0065] The operating system file further includes code 
implementing a Java (tm) virtual machine and a graph- 
ics layer known as RTX X-Windows server (tm) by Vlsi- 
com of San Diego, California. The Java virtual machine 
includes core Java packages including Java.lang, Java, 
io, Java.applet, Java.net and Java.awt which in connec- 
tion with the operating system files create a Java run 
time environment at the system microprocessor 90 
shown in Figure 3. 

[0066] Still referring to Figure 5, the system subdirec- 
tory 167 includes an applications subdirectory 175. The 
applications subdirectory 1 75 includes a telephone sub- 
directory 1 79 containing files for directing the system mi- 
croprocessor 90 to cooperate with the telephony micro- 
controller 92 to provide telephony functionality. The ap- 
plications subdirectory further includes an event parser 
subdirectory 181 containing event parser files, a main 
event handler subdirectory 183 containing main event 
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handler files, a browser subdirectory 185 containing 
browser files, a public interface subdirectory 187 con- 
taining public interface method files 189 and a handler 
subdirectory 191 containing handler files. 
[0067] The data subdirectory 1 69 includes a configu- 
ration data subdirectory 207 containing configuration 
files including a CPE ID file 210 and an event code 
lookup table file 208. The CPE ID file 210 holds a CPE 
ID of the telephone, to identify the telephone from other 
telephones which may be connected to the same tele- 
phone line. The event code lookup table file 208 con- 
tains an event code lookup table having parameter type 
and URL event code fields 21 1 and 21 3 respectively, for 
associating parameter types with URL event codes. 
[0068] The Java subdirectory 1 71 includes Java class 
files of the type classes.jar. 

[0069] The temporary subdirectory 173 is used to 
store applets, HTML pages and files which may be 
downloaded over one of the analog subscriber lines 18 
and 28 shown.in Figure 3. . _ . 

Figure 6 

[0070] Referring to Figure 6, the RAM 158 is used to 
maintain an FSK receive buffer 200, a variable buffer 
202, a main event handler buffer 204, an event queue 
buffer 206 and a routing table 1 55. 
[0071] The FSK receive buffer 200 is of sufficient 
length to receive FSK messages, in general. In this em- 
bodiment, such messages are of the form shown in Fig- 
ure 2. 

[0072] Referring back to Figure 6, the variable buffer 
202 and the main event handler buffer 204save hold var- 
iable and main event records respectively. The variable 
records include a parameter type field 21 5 and a data 
field 216 and the main event records include a param- 
eter type field 217 and a data field 209. The event queue 
buffer 206 is a variable length buffer for storing various 
event queue records, each of which includes an event 
code field 248 ancLa data field 250. . . ^ 

Figure 7 

[0073] Referring briefly to Figure 3, the system micro- 
processor 90, FLASH memory 156, and RAM 158 to- 
gether form a computer architecture. Referring to Figure 
7, this architecture is shown generally at 162. 

Figure 8 

[0074] Referring to Figure 8, the event parser files in 
the event parser directory establish an event parser 
thread running on the system processor, as shown gen- 
erally at 220. The event parser thread includes a read 
comm port instance 222 which directs the system mi- 
croprocessor 90 shown in Figure 3, to communicate with 
the telephony microcontroller to read the FSK receiver 
1 50 to determine whether or not an FSK message is be- 



ing received at the FSK receiver 150. If a message is 
not being received, the event parser thread is blocked 
on the read comm port instance 222. If the FSK receiver 
150 is receiving an FSK message, such message is 
5 passed to the system microprocessor and stored in the 
FSK receive buffer 200. 

[0075] The event parser thread then includes a check- 
sum instance 224 which directs the processor to read 
the contents of the FSK receive buffer 200 to determine 
10 whether or not the message was received properly. If 
the message was not received properly, it is ignored and 
the processor is directed back to the read comm port 
instance 222. 

[0076] If the message was received properly, the 
15 processor is directed to a CPE ID match instance 226 
in which the contents of the CPE ID register 210 stored 
in the FLASH memory 1 56 is compared against the con- 
tents of the CPE ID field of the FSK message stored in 
the FSK receive buffer 200. If the CPE IDs do not match, 
20, .the processor is retu med.to the read comm port instance 
222 

[0077] If the CPE IDs match, the processor is directed 
to a store parameter type instance 232 which directs the 
processor to store the contents of the parameter type 

25 field and the contents of the data field in the variable 
buffer 202. The system microprocessor 90 is then di- 
rected to a copy instance 234 which directs it to copy 
the contents of the variable buffer 202 to the main event 
handler buffer 204. The processor is then returned back 

30 to the read comm port instance 222 where it blocks until 
a further FSK message is received. 

Figure 9 

35 Main event handler thread 

[0078] Referring to Figure 9, the main event handler 
files in the main event handler subdirectory 183 estab- 
lish a main event handler thread as shown generally at 

40__24O. The main event hand!sr-thre?d includes a buffer^ 
loaded instance 242 which directs the system micro- 
processor 90 to determine whether or not the main event 
handler buffer 204 shown in Figure 6 is full. If this buffer 
is not loaded then the processor blocks on the buffer 

45 loaded instance 242. 

[0079] If the main event handler buffer is loaded, the 
processor is directed to a lookup table instance 244 
which directs the processor to find an event code record 
in the event code lookup table 208 shown in Figure 5, 

so having parameter type field 211 contents corresponding 
to the contents of the parameter type field in the main 
event handler buffer 204 shown in Figure 6. 
[0080] On finding such an event code record, the sys- 
tem microprocessor 90 is directed to a copy event code 

55 instance 246 which directs the processor to copy the 
contents of the event code field 21 3 from the event code 
record into the event queue buffer 206 shown in Figure 
6 and also to copy the contents of the data field 209 of 
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the main event handler buffer 204 shown in Figure 6 to 
the event queue buffer 206 in association with the event 
code. Thus, the processor stores, in the event queue 
buffer 206, a first event record including an event code 
from the event code lookup table 208 and data from the 
data field 250 of the main event handler buffer 204 
shown in Figure 6. 

Figure 10 

[0081] Referring to Figure 10, the event dispatcher 
files in the main event handler subdirectory 183 shown 
in Figure 5 establish an event dispatcher thread as 
shown generally at 180. The event dispatcher thread in- 
cludes a first instance which directs the system proces- 
sor to address the next event code record in the event 
queue buffer 206 shown in Figure 6. The system proc- 
essor is then directed to instance 184 which directs it to 
determine which threads have registered for the event 

code record addressed at instance.182._The registration 

of threads with event code records is assumed to have 
been previously accomplished according to convention- 
al practices. After having determined which threads 
have registered for the current event code record, in- 
stance 186 directs the processor to communicate the 
current event code record from the event queue buffer 
206 to the registered threads. In this manner, data is 
passed from the event queue buffer 206 to appropriate 
threads. 

Figure 11 

[0082] Referring to Figure 11, a browser thread 260 
is implemented by the browser files in the browser sub- 
directory 185 shown in Figure 5. The browser thread in- 
cludes a pend instance 262 which directs the processor 
to pend for receipt of a URL event code record from the 
event dispatcher thread 180 shown in Figure 10. 
[0083] On receipt of a URL event code record, a re- 
. quest instance 264 djrects the system microprocessor 
90 to request a connection to a world-wide web resource 
specified by the URL in the data field 250 of the event 
record. Such connection is requested using classes in 
the Java net package of the Java virtual machine. 

Figure 12 

[0084] Referring to Figure 12, if a connection is re- 
quested by an application, the network driver 270 includ- 
ed within the operating system file 165 shown in Figure 
5 is run. The network driver task includes a first instance 
272 which directs the system microprocessor 90 to pend 
awaiting a connection request from the browser thread 
shown in Figure 11. 

[0085] On receiving a connection request, instance 
274 directs the system microprocessor 90 to check the 
routing table 155 shown in Figure 6 to determine wheth- 
er or not a network connection has already been estab- 



lished. Referring to Figure 12, if no such connection has 
been established, instance 276 directs the system mi- 
croprocessor 90 to command the modem 130 to con- 
nect to the third telephone line 58 to place a telephone 
s call on the third telephone line 28 via public switched 
telephone network to the Internet services provider 24 
shown in Figure 1 . 

[0086] Referring back to Figure 12, establishment of 
a connection with the Internet services provider involves 

10 a handshaking instance 278. 

[0087] On establishing connection with the Internet 
services provider, instance 280 directs the processor to 
negotiate a point to point protocol (PPP) connection with 
the Internet services provider and to store the details of 

15 the PPP connection in the routing table 1 55. A network 
connection is thus created. 

[0088] Instance 282 then directs the system micro- 
processor 90 to notify the requesting application of the 
connection to the resource specified by the URL in the 

20 data field 250 of the event record. 

[0089] On completion of transmission of the details to 
the browser thread, the system microprocessor 90 is di- 
rected back to instance 272 where it pends on receipt 
of a further connection request from the browser thread. 

25 [0090] It should be noted that at instance 274, where 
the processor is directed to determine whether or not a 
connection has already been established, if such a con- 
nection has already been established, the processor is 
directed directly to instance 282 where it notifies the re- 

30 questing application of such connection. 

[0091] Referring back to Figure 11, notification of the 
connection established by the network driver task 
shown in Figure 12 is received at instance 290 in Figure 
11 

35 [0092] Still referring to Figure 11 , if noconnection has 
been made and no connection can be established, in- 
stance 292 directs the system microprocessor 90 to dis- 
play an error banner to indicate that the connection 
could not be established. The processor is then returned 

40 b3ck to the pend instance 26?, . 

[0093] If at instance 290, notification of a connection 
has been received, instance 294 directs the processor 
to receive and decode an HTML file from the resource 
indicated by the URL passed to the browser by the event 

45 dispatcher. In this embodiment, the HTML page is down- 
loaded from the first content provider 26. The HTML file 
is received and stored in the temporary directory 173 in 
the FLASH 156. 

[0094] Instance 295 then directs the processor to de- 
termine whether or not the HTML file has been success- 
fully decoded, according to well known browser decod- 
ing criteria. In this embodiment, the browser includes a 
security manager which imposes restrictions on what 
hosts any applets within the HTML file can communicate 
55 with. In particular in this embodiment, applets are re- 
stricted from reading or writing to the flash memory 
shown in Figure 5. Applets may only open socket con- 
nections to the host that served them. They are not per- 
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mitted to open server sockets themselves, they are not 
permitted to start other processes on the local host and 
they are not permitted to have native methods. 
[0095] If the HTML file was not successfully decoded, 
the processor is directed to the error banner instance 5 
292. If the HTML file was successfully decoded, in- 
stance 296 directs the processor to execute the HTML 
file to display the content contained therein and to 
spawn threads on each applet, if any, contained therein. 
After spawning such threads, the processor is returned 10 
to the pend instance 262. 

Operation 

[0096] Referring to Figure 1 , the content provider 26 is 
includes a web server 350 and a database of HTML files 
352. The database 36 at the personal computer of the 
first CPE has a list of URLs which identify the individual 
HTML pages 352 at the first content provider 26. Thus, 
the personal computer at the first CPE acts as. means .20 
for maintaining a list of network resource locators at the 
first customer premises equipment and selectively using 
the network resource locators in respective FSK mes- 
sages. 

[0097] It is contemplated that a user of the second tel- 2s 
ephone 20 makes a telephone call on the second tele- 
phone line 18 to the first telephone 32. Thus the second 
telephone acts as means disposed at the second cus- 
tomer premises equipment for establishing a telephone 
call on a first telephone line, between the first and sec- so 
ond customer premises equipment. When a call has 
been established, during the call, the user of the second 
telephone may request to a user of the first telephone, 
by voice, to see a picture of services provided by the 
user of the first telephone. In this case, the user of the 35 
first telephone enters on the keyboard 42 an indication 
of the desired page to be shown and the microprocessor 
34 retrieves a corresponding associated URL from the 
memory 36. The microprocessor then produces a series 
of bytes representing a desired FSK message to be sent 40 
to the second telephone and presents such bytes along 
with appropriate command codes to the FSK transmit- 
ter, while at the same time setting a mute signal active 
to activate the mute circuit 40 to prevent FSK signals 
produced by the FSK transmitter from being transmitted 
to the telephone 32. The FSK signals are, however, 
transmitted on the first telephone line 16 to the central 
office 22 which further transmits these signals to the 
second telephone via the second telephone line 18. 
Thus, the microprocessor and FSK transmitter act as a so 
processor transmitter or means for generating at the first 
customer premises equipment a frequency shift keyed 
message containing a network resource locator. The 
mute circuit 40 and processor act as means for trans- 
mitting the FSK message from the first customer premis- & 
es equipment to the second customer premises equip- 
ment during the telephone call. 
[0098] The second telephone line 18 receives the 



FSK signals through the FSK receiver 150 and deposits 
a replica of the FSK message in the FSK receive buffer 
200. The FSK receiver thus acts as a receiver or means 
for receiving the FSK message at the second customer 
premises equipment. 

[0099] The telephone further receives the FSK mes- 
sage 70 through the event parser thread 220 shown in 
Figure 8 and the main event handler thread 240 shown 
in Figure 9 which load the event queue buffer 206 with 
a URL event code and the contents of the parameter 
data field 82 which contains the URL of the HTML file 
356 at the first content provider 26 shown in Figure 1. 
Thus, the processor acts as means for extracting the 
network resource locator from the FSK message. 
[0100] Referring to Figures 1 and 11, the browser 
thread 260 directs the system microprocessor 90 to con- 
trol the modem 130 to establish a connection with the 
Internet services provider 24 and the network driver 270 
shown in Figure 12 establishes a network connection 
-between the telephone 20.and.the first content provider 
26 through the third telephone line 28, the Internet serv- 
ices provider 24 and the world-wide web 23. Thus, the 
system processor and browser thread act as means for 
establishing a connection to a network resource indicat- 
ed by the network resource locator and more particular- 
ly, means for establishing the connection to the network 
resource on a second telephone line while maintaining 
the telephone call active on the first telephone line. 
[0101] The first content provider 26 transmits the 
HTML file containing the applet back to the telephone 
20 and such HTML file is received in the temporary di- 
rectory 173 shown in Figure 5. Thus, the processor and 
browser act as means for downloading a file from the 
network resource identified by the network resource lo- 
cator. The browser thread 260 then executes the HTML 
file and displays any content on the display 102 and 
spawns applet threads on each applet included therein. 
[0102] In this embodiment, the entire content of the 
HTML file from the content provider is displayed on the 
display 102. Thus, marketing or advertising content is- 
automatically displayed on the display. The browser has 
no navigation tools or buttons and, therefore, the adver- 
tisement is displayed unobstructed. 
[0103] The HTML file may include a plurality of links 
which the user can select using an input device (not 
shown) and such links may be used by the browser to 
connect to another resource on the world-wide web or 
to connect to a resource within the telephone itself. Such 
resource would be stored in a sub-directory of the sys- 
tem subdirectory 167 shown in Figure 5. 
[0104] Applets may be used to create animated mar- 
keting advertisements or presentations which may be 
more appealing to a user. In addition, high resolution 
pictures may be included within the HTML file as GIF 
files or the like. 

[01 05] As HTML pages are for use on display screens 
of a predefined aspect ratio, advertising for use on tel- 
ephones of the type described, can be simply and easily 
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d) receiving said FSK message at said second 
customer premises equipment. 

2. A method as claimed in claim 1 further including ex- 
5 tracting said network resource locator from said 

FSK message. 

3. A method as claimed in claim 2 further including es- 
tablishing a connection to a network resource indi- 

10 cated by said network resource locator 

4. A method as claimed in claim 3 further including es- 
tablishing said connection to said network resource 
using a second telephone line. 

15 

5. A method as claimed in claim 4 further including es- 
tablish ing said connection to said network resource 
while maintaining said tetephone call active. 

20 6. A method as claimed in claim 5 further including 
downloading a file from said network resource indi- 
cated by said network resource locator. 

7. A method as claimed in claim 6 further including 
25 producing a display image from said file at said sec- 
ond customer premises equipment, while display- 
ing an associated image at said first customer 
premises equipment. 

30 8. A method as claimed in claim 7 wherein producing 
includes producing said display image at said sec- 
ond customer premises equipment while displaying 
said associated image at said first customer premis- 
es equipment, white maintaining said telephone call 

35 to enable a user of said second customer premises 
equipment to view images designated by a user of 
said first customer premises equipment, while con- 
ducting a telephone conversation with said user of 
said first customer premises equipment. 

AO . . ... 

9. A method as claimed in claim 1 further including 
maintaining a list of network resource locators at 
said first customer premises equipment and selec- 
tively using said network resource locators in re- 

45 spective FSK messages: 



developed using conventional HTML page development 
techniques. This facilitates convenient creation of ad- 
vertising content for the telephone. 
[0106] it will be appreciated that the microprocessor 
at the first customer premises equipment directs the 
mute circuit 40 to perform the mute function only where 
the FSK message is being transmitted by the FSK trans- 
mitter. Otherwise, the mute circuit 40 is not active and 
users of the first and second telephones may continue 
to converse, while the second telephone navigates to 
the network resource specified by the URL in the FSK 
message transmitted by the second telephone. Thus, 
the browser and processor at the second telephone and 
the personal computer at the first customer premises 
equipment act as means for producing a display image 
at the second customer premises equipment, from the 
file, while displaying an associated image at the first cus- 
tomer premises equipment. More particularly, the appa- 
ratus includes means for producing a display image at 
the second customer premises equipment while dis- 
playing an associated image at the first customer 
premises equipment, while maintaining the telephone 
call on the first telephone line to enable a user of the 
second customer premises equipment to view images 
designated by a user of the first customer premises 
equipment, while conducting a telephone conversation 
with the user of the first customer premises equipment. 
[0107] While the second telephone navigates to the 
network resource specified by the URL in the FSK mes- 
sage and while the voice call is being conducted, the 
first computer produces display images at the first cus- 
tomer, premises equipment, the display images corre- 
sponding to respective network resource locators in- 
cluded in respectively transmitted FSK messages. 
[0108] While specific embodiments of the invention 
have been described and illustrated, such embodiments 
should be considered illustrative of the invention only 
and not as limiting the invention as construed in accord- 
ance with the accompanying claims. 



Claims 

1. A method of conducting a multimedia telephone 
call, the method comprising: 

a) establishing a telephone call between first 
and second customer premises equipment; 

b) generating, at said first customer premises 
equipment, a frequency shift keyed (FSK) mes- 
sage containing a network resource locator; 

c) transmitting said FSK message from said 
first customer premises equipment to said sec- 
ond customer premises equipment during said 
telephone call; and 



1 0. A method as claimed in claim 9 further including as- 
sociating image files with respective said network 
resource locators and producing display images 

so corresponding to respective network resource loca- 
tors when an FSK message is generated. 

11. A system for conducting a multimedia telephone 
call, the system comprising: 

55 

a) first and second customer premises equip- 
ment and means disposed at at least one of 
said first and second customer premises equip- 



10 



19 



EP0 971 513 A2 



20 



ment for establishing a telephone call on a first 
telephone line, between said first and second 
customer premises equipment; 

b) means for generating, at said first customer 
premises equipment, a frequency shift keyed 
(FSK) message containing a network resource 
locator; 

c) means for transmitting said FSK message 
from said first customer premises equipment to 
said second customer premises equipment 
during said telephone call; and 

d) means for receiving said FSK message at 
said second customer premises equipment. 

12. A system as claimed in claim 11 further including 
means for extracting said network resource locator 

from said FSK-message. 

13. A system as claimed in claim 12 further including 
means for establishing a connection to a network 
resource indicated by said network resource loca- 
tor 

14. A system as claimed in claim 13 further including 
means for establishing said connection to said net- 
work resource on a second telephone line. 

15. A system as claimed in claim 14 further including 
means for establishing said connection to said net- 
work resource while maintaining said telephone call 
active on said first telephone line. 

16. A system as claimed in claim 15 further including 
means for downloading a file from said network re- 
source indicated by said network resource locator. 

17. A system as claimed In claim 16 further including 
means for producing a display image from said file 
at said second customer premises equipment, while 
displaying an associated image at said first custom- 
er premises equipment. 

18. A system as claimed in claim 17 wherein said 
means for producing includes means for producing 
said display image at said second customer premis- 
es equipment while displaying said associated im- 
age at said first customer premises equipment, 
while maintaining said telephone call on said first 
telephone line to enable a user of said second cus- 
tomer premises equipment to view images desig- 
nated by a user of said first customer premises 
equipment, while conducting a telephone conversa- 
tion with said user of said first customer premises 
equipment. 



19. A system as claimed in claim 11 further including 
means for maintaining a list of network resource lo- 
cators at said first customer premises equipment 
and means for selectively using said network re- 

5 source locators in respective FSK messages. 

20. A system as claimed in claim 19 further including 
means for associating image files with respective 
said network resource locators and means for pro- 

io ducing display images at said first customer premis- 
es equipment, said display images corresponding 
to respective network resource locators when an 
FSK message is generated. 

is 21. A system for conducting a multimedia telephone 
call, the system comprising: 

a) first and second customer premises equip- 
ment operable to establish a telephone call be- 

20 tween each-other on a first telephone line; 

b) a first processor at said first customer 
premises equipment, for generating at said first 
customer premises equipment, a frequency 

25 shift keyed (FSK) message containing a net- 

work resource locator; 

c) a transmitter for transmitting said FSK mes- 
sage from said first customer premises equip- 

30. ment to said second customer premises equip- 

ment during said telephone call; and 

d) a receiver at said second customer premises 
equipment for receiving said FSK message. 

35 

22. A system as claimed in claim 21 further including a 
second processor at said second customer premis- 
es equipment for extracting said network resource 
locator from said FSK message. 

40 - - - 

23. A system as claimed in claim 22 further including a 
browser running on said second processor at said 
second customer premises equipment for estab- 
lishing a connection to a network resource indicated 

45 by said network resource locator. 

24. A system as claimed in claim 23 further including a 
modem for establishing said connection to said net- 
work resource on a second telephone line. 

50 

25. A system as claimed in claim 24 wherein said sec- 
ond processor at said second customer premises 
equipment is operable to establish said connection 
to said network resource while maintaining said tel- 

55 ephone call active on said first telephone line. 

26. A system as claimed in claim 25 wherein said sec- 
ond processor and said browser at said second cus- 
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tomer premises equipment are operable to down- 
load a file from said network resource indicated by 
said network resource locator. 

27. A system as claimed in claim 26 further including 
first and second displays at said first and second 
customer premises equipment respectively and 
wherein said second processor at said second cus- 
tomer premises equipment is operable to cooperate 
with said second display to produce a display image 
from said file at said second customer -premises 
equipment while said first processor at said first 
customer premises equipment is operable to coop- 
erate with said first display to display an associated 
image at said first customer premises equipment. 

28. A system as claimed in claim 27 wherein said sec- 
ond processor is operable to cooperate with said 
second display to produce said display image while 
said first processor is operable to cooperate with 
said first display to display said associated image, 
while maintaining said telephone call on said first 
telephone line to enable a user of said second cus- 
tomer premises equipment to view images desig- 
nated by a user of said first customer premises 
equipment, while conducting a telephone conversa- 
tion with said user of said first customer premises 
equipment. 

29. A system as claimed in claim 21 further including 
memory at said first customer premises equipment 
for maintaining a list of network resource locators 
at said first customer premises equipment, said first 
processor being programmed to selectively include 
said network resource locators in respective FSK 
messages. 

30. A system as claimed in claim 29 wherein said first 
processor is programmed to associate image files 
with respectjye said network resource locators and 
produce display images at said first customer 
premises equipment, said display images corre- 
sponding to respective network resource locators 
included in said FSK messages, as said FSK mes- 
sages are generated. 

31. Cancelled ] 

32. A method as claimed in claim 40 further including 
maintaining a list of network resource locators at 
said first customer premises equipment and selec- 
tively using said network resource locators in re- 
spective FSK messages. 

33. A method as claimed in claim 32 further including 
associating image files with respective said network 

. resource locators and producing display images 
corresponding to respective network resource loca- 



tors when an FSK message is generated. 

34. Cancelled.] 

s 35. An apparatus as claimed in claim 41 further includ- 
ing means for maintaining a list of network resource 
locators at said first customer premises equipment 
and selectively, using said network resource loca- 
tors in respective FSK messages. 

10 

36. An apparatus as claimed in claim 41 further includ- 
ing means for associating image files with respec- 
tive said network resource locators and for produc- 
ing display images at said first customer premises 

'5 equipment, said display images corresponding to 
respective network resource locators when an FSK 
message, is generated. 

37. Cancelled.] 

20 

38. An apparatus as claimed in claim 42 further includ- 
ing memory at said first customer premises equip- 
ment for maintaining a list of network resource lo- 
cators at said first customer premises equipment, 

25 and wherein said processor is programmed to se- 
lectively include said network resource locators in 
respective FSK messages. 

39. An apparatus as claimed in claim 38 wherein said 
30, processor is programmed to associate image files 

with respective said network resource locators and 
produce display images at said first customer 
premises equipment, said display images corre- 
sponding to respective network resource locators 
35 included in said FSK messages, as said FSK mes- 
sages are generated. 

40. A method of dynamically dispatching network re- 
source locators, the method comprising: 

40 

a) generating at a first customer premises 
equipment a frequency shift keyed (FSK) mes- 
sage representing a network resource locator; 
and 

45 

b) transmitting said FSK message from said 
first customer premises equipment to a second 
customer premises equipment during a tele- 
phone call between said first customer premis- 

50 es equipment and the second customer 

premises equipment. 

41. An apparatus for dynamically dispatching network 
resource locators from a first customer premises 

. 55 equipment to a second customer premises equip- 
ment, the apparatus comprising: 

a) means for generating at the first customer 
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premises equipment a frequency shift keyed 
(FSK) message representing a network re- 
source locator; and 

t>) means for transmitting said FSK message s 
from the first customer premises equipment to 
the second customer premises equipment dur- 
ing a telephone call between the first customer 
premises equipment and the second customer 
premises equipment. 10 

42. An apparatus for dynamically dispatching network 
resource locators from a first customer premises 
equipment to a second customer premises equip- 
ment, the apparatus comprising: is 

a) a processor at the first customer premises 
equipment, for generating a frequency shift 
keyed (FSK) message containing a network re- 
source locator; and 20 

b) a transmitter for transmitting said FSK mes- 
sage from the first customer premises equip- 
ment to the second customer premises equip- 
ment during a telephone call between the first 25 
customer premises equipment and the second 
customer premises equipment. 
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